Ayush Kaushal

What do tokens know about their characters and how do they know it?

2022 · arXiv (Cornell University)

Pre-trained language models (PLMs) that use subword tokenization schemes can succeed at a variety of language tasks that require character-level information, despite lacking explicit access to the character composition of tokens. Here, studying a range …