Artem Mukhin

Expanding the Context of Large Language Models Via Linear Interpolation of Positional Embeddings

2024

This study explores the problem of limited context in Russian large language models (LLMs) and proposes a new method for increasing their context size in an efficient manner. The proposed method is based on the …