Jaewoong Cho

Mini-Batch Optimization of Contrastive Loss

2023 · arXiv (Cornell University)

Contrastive learning has gained significant attention as a method for self-supervised learning. The contrastive loss function ensures that embeddings of positive sample pairs (e.g., different samples from the same class or different views of the …

Lexico: Extreme KV Cache Compression via Sparse Coding over Universal Dictionaries

2024 · arXiv (Cornell University)

We introduce Lexico, a novel KV cache compression method that leverages sparse coding with a universal dictionary. Our key finding is that key-value cache in modern LLMs can be accurately approximated using sparse linear combination …