Scao, Teven Le

HuggingFace's Transformers: State-of-the-art Natural Language Processing

2019 · arXiv (Cornell University)

Recent progress in natural language processing has been driven by advances in both model architecture and model pretraining. Transformer architectures have facilitated building higher-capacity models and pretraining has made it possible to effectively utilize this …

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

2022 · arXiv (Cornell University)

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed …