Tatsuya Hiraoka

Repetition Neurons: How Do Language Models Produce Repetitions?

2024 · arXiv (Cornell University)

This paper introduces repetition neurons, regarded as skill neurons responsible for the repetition problem in text generation tasks. These neurons are progressively activated more strongly as repetition continues, indicating that they perceive repetition as a …