Steven Abreu

Mamba-PTQ: Outlier Channels in Recurrent Large Language Models

2024 · arXiv (Cornell University)

Modern recurrent layers are emerging as a promising path toward edge deployment of foundation models, especially in the context of large language models (LLMs). Compressing the whole input sequence in a finite-dimensional representation enables recurrent …