Shijie Wu

Are All Languages Created Equal in Multilingual BERT?

2020

Multilingual BERT (mBERT) However, these evaluations have focused on cross-lingual transfer with highresource languages, covering only a third of the languages covered by mBERT. We explore how mBERT performs on a much wider set of …

How Do Multilingual Encoders Learn Cross-lingual Representation?

2022 · arXiv (Cornell University)

NLP systems typically require support for more than one language. As different languages have different amounts of supervision, cross-lingual transfer benefits languages with little to no training data by transferring from other languages. From an …

Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT

2019

Shijie Wu, Mark Dredze. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Emerging Cross-lingual Structure in Pretrained Language Models

2020

We study the problem of multilingual masked language modeling, i.e. the training of a single model on concatenated text from multiple languages, and present a detailed study of several factors that influence why these models …

BloombergGPT: A Large Language Model for Finance

2023 · arXiv (Cornell University)

The use of NLP in the realm of financial technology is broad and complex, with applications ranging from sentiment analysis and named entity recognition to question answering. Large Language Models (LLMs) have been shown to …