Chenkai Sun

Word Embeddings Are Steers for Language Models

2023 · arXiv (Cornell University)

Language models (LMs) automatically learn word embeddings during pre-training on language corpora. Although word embeddings are usually interpreted as feature vectors for individual words, their roles in language model generation remain underexplored. In this work, …