Vasileios Kalantzis

Multi-Sense Embeddings for Language Models and Knowledge Distillation

2025 · arXiv (Cornell University)

Transformer-based large language models (LLMs) rely on contextual embeddings which generate different (continuous) representations for the same token depending on its surrounding context. Nonetheless, words and tokens typically have a limited number of senses (or …