Prajwal Singhania

Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs

2024 · arXiv (Cornell University)

Large language models can memorize and repeat their training data, causing privacy and copyright risks. To mitigate memorization, we introduce a subtle modification to the next-token training objective that we call the goldfish loss. During …