J. Hyun Kim

Lexico: Extreme KV Cache Compression via Sparse Coding over Universal Dictionaries

2024 · arXiv (Cornell University)

We introduce Lexico, a novel KV cache compression method that leverages sparse coding with a universal dictionary. Our key finding is that key-value cache in modern LLMs can be accurately approximated using sparse linear combination …